
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
  <head>
    <meta http-equiv="Content-Type" content="text/html; charset=utf-8">
    <link href='/css/styles.css' rel='stylesheet' type='text/css' />
    <link href='/images/favicon.png' rel='shortcut icon' />
    <script src='/js/jquery.min.1.4.js'></script>
    <script src='/js/app.js'></script>
    <meta content='width=device-width, minimum-scale=1.0, maximum-scale=1.0' name='viewport' />
    <title>Redis中文资料站</title>
	<meta http-equiv="description" content="redis中文资料站，下载安装redis，查找redis常用命令（commands），选择适合的redis客户端方式，配置redis主从（master-slave），阅读redis官方文档，社区里了解更多redis信息，提交redis的bug。" />
  </head>
  <body class=''>
    <script src='/js/head.js'></script>
    <div class='text'>
      <article id='topic'>
        <h1>Redis Persistence</h1>
        
        <p>Redis provides a different range of persistence options:</p>
        
        <ul>
        <li>The RDB persistence performs point-in-time snapshots of your dataset at specified intervals.</li>
        <li>the AOF persistence logs every write operation received by the server, that will be played again at server startup, reconstructing the original dataset. Commands are logged using the same format as the Redis protocol itself, in an append-only fashion. Redis is able to rewrite the log on background when it gets too big.</li>
        <li>If you wish, you can disable persistence at all, if you want your data to just exist as long as the server is running.</li>
        <li>It is possible to combine both AOF and RDB in the same instance. Notice that, in this case, when Redis restarts the AOF file will be used to reconstruct the original dataset since it is guaranteed to be the most complete.</li>
        </ul>
        
        
        <p>The most important thing to understand is the different trade-offs between the
        RDB and AOF persistence. Let's start with RDB:</p>
        
        <h2>RDB advantages</h2>
        
        <ul>
        <li>RDB is a very compact single-file point-in-time representation of your Redis data. RDB files are perfect for backups. For instance you may want to archive your RDB files every hour for the latest 24 hours, and to save an RDB snapshot every day for 30 days. This allows you to easily restore different versions of the data set in case of disasters.</li>
        <li>RDB is very good for disaster recovery, being a single compact file can be transfered to far data centers, or on Amazon S3 (possibly encrypted).</li>
        <li>RDB maximizes Redis performances since the only work the Redis parent process needs to do in order to persist is forking a child that will do all the rest. The parent instance will never perform disk I/O or alike.</li>
        <li>RDB allows faster restarts with big datasets compared to AOF.</li>
        </ul>
        
        
        <h2>RDB disadvantages</h2>
        
        <ul>
        <li>RDB is NOT good if you need to minimize the chance of data loss in case Redis stops working (for example after a power outage). You can configure different <em>save points</em> where an RDB is produced (for instance after at least five minutes and 100 writes against the data set, but you can have multiple save points). However you'll usually create an RDB snapshot every five minutes or more, so in case of Redis stopping working without a correct shutdown for any reason you should be prepared to lose the latest minutes of data.</li>
        <li>RDB needs to fork() often in order to persist on disk using a child process. Fork() can be time consuming if the dataset is big, and may result in Redis to stop serving clients for some millisecond or even for one second if the dataset is very big and the CPU performance not great. AOF also needs to fork() but you can tune how often you want to rewrite your logs without any trade-off on durability.</li>
        </ul>
        
        
        <h2>AOF advantages</h2>
        
        <ul>
        <li>Using AOF Redis is much more durable: you can have different fsync policies: no fsync at all, fsync every second, fsync at every query. With the default policy of fsync every second write performances are still great (fsync is performed using a background thread and the main thread will try hard to perform writes when no fsync is in progress.) but you can only lose one second worth of writes.</li>
        <li>The AOF log is an append only log, so there are no seeks, nor corruption problems if there is a power outage. Even if the log ends with an half-written command for some reason (disk full or other reasons) the redis-check-aof tool is able to fix it easily.</li>
        <li>Redis is able to automatically rewrite the AOF in background when it gets too big. The rewrite is completely safe as while Redis continues appending to the old file, a completely new one is produced with the minimal set of operations needed to create the current data set, and once this second file is ready Redis switches the two and starts appending to the new one.</li>
        <li>AOF contains a log of all the operations one after the other in an easy to understand and parse format. You can even easily export an AOF file. For instance even if you flushed everything for an error using a FLUSHALL command, if no rewrite of the log was performed in the meantime you can still save your data set just stopping the server, removing the latest command, and restarting Redis again.</li>
        </ul>
        
        
        <h2>AOF disadvantages</h2>
        
        <ul>
        <li>AOF files are usually bigger than the equivalent RDB files for the same dataset.</li>
        <li>AOF can be slower then RDB depending on the exact fsync policy. In general with fsync set to <em>every second</em> performances are still very high, and with fsync disabled it should be exactly as fast as RDB even under high load. Still RDB is able to provide more guarantees about the maximum latency even in the case of an huge write load.</li>
        <li>In the past we experienced rare bugs in specific commands (for instance there was one involving blocking commands like BRPOPLPUSH) causing the AOF produced to don't reproduce exactly the same dataset on reloading. This bugs are rare and we have tests in the test suite creating random complex datasets automatically and reloading them to check everything is ok, but this kind of bugs are almost impossible with RDB persistence. To make this point more clear: the Redis AOF works incrementally updating an existing state, like MySQL or MongoDB does, while the RDB snapshotting creates everything from scratch again and again, that is conceptually more robust. However 1) It should be noted that every time the AOF is rewritten by Redis it is recreated from scratch starting from the actual data contained in the data set, making resistance to bugs stronger compared to an always appending AOF file (or one rewritten reading the old AOF instead of reading the data in memory). 2) We never had a single report from users about an AOF corruption that was detected in the real world.</li>
        </ul>
        
        
        <h2>Ok, so what should I use?</h2>
        
        <p>The general indication is that you should use both persistence methods if
        you want a degree of data safety comparable to what PostgreSQL can provide you.</p>
        
        <p>If you care a lot about your data, but still can live with a few minutes of
        data lose in case of disasters, you can simply use RDB alone.</p>
        
        <p>There are many users using AOF alone, but we discourage it since to have an
        RDB snapshot from time to time is a great idea for doing database backups,
        for faster restarts, and in the event of bugs in the AOF engine.</p>
        
        <p>Note: for all this reasons we'll likely end unifying AOF and RDB into a single persistence model in the future (long term plan).4</p>
        
        <p>The following sections will illustrate a few more details about the two persistence models.</p>
        
        <p><a name="snapshotting"></a></p>
        
        <h2>Snapshotting</h2>
        
        <p>By default Redis saves snapshots of the dataset on disk, in a binary
        file called <code>dump.rdb</code>. You can configure Redis to have it save the
        dataset every N seconds if there are at least M changes in the dataset,
        or you can manually call the <a href="/commands/save">SAVE</a> or <a href="/commands/bgsave">BGSAVE</a> commands.</p>
        
        <p>For example, this configuration will make Redis automatically dump the
        dataset to disk every 60 seconds if at least 1000 keys changed:</p>
        
        <pre><code>save 60 1000&#x000A;</code></pre>
        
        <p>This strategy is known as <em>snapshotting</em>.</p>
        
        <h3>How it works</h3>
        
        <p>Whenever Redis needs to dump the dataset to disk, this is what happens:</p>
        
        <ul>
        <li><p>Redis <a href="http://linux.die.net/man/2/fork">forks</a>. We now have a child
        and a parent process.</p></li>
        <li><p>The child starts to write the dataset to a temporary RDB file.</p></li>
        <li><p>When the child is done writing the new RDB file, it replaces the old
        one.</p></li>
        </ul>
        
        
        <p>This method allows Redis to benefit from copy-on-write semantics.</p>
        
        <p><a name="append-only-file"></a></p>
        
        <h2>Append-only file</h2>
        
        <p>Snapshotting is not very durable. If your computer running Redis stops,
        your power line fails, or you accidentally <code>kill -9</code> your instance, the
        latest data written on Redis will get lost.  While this may not be a big
        deal for some applications, there are use cases for full durability, and
        in these cases Redis was not a viable option.</p>
        
        <p>The <em>append-only file</em> is an alternative, fully-durable strategy for
        Redis.  It became available in version 1.1.</p>
        
        <p>You can turn on the AOF in your configuration file:</p>
        
        <pre><code>appendonly yes&#x000A;</code></pre>
        
        <p>From now on, every time Redis receives a command that changes the
        dataset (e.g. <a href="/commands/set">SET</a>) it will append it to the AOF.  When you restart
        Redis it will re-play the AOF to rebuild the state.</p>
        
        <h3>Log rewriting</h3>
        
        <p>As you can guess, the AOF gets bigger and bigger as write operations are
        performed.  For example, if you are incrementing a counter 100 times,
        you'll end up with a single key in your dataset containing the final
        value, but 100 entries in your AOF. 99 of those entries are not needed
        to rebuild the current state.</p>
        
        <p>So Redis supports an interesting feature: it is able to rebuild the AOF
        in the background without interrupting service to clients. Whenever
        you issue a <a href="/commands/bgrewriteaof">BGREWRITEAOF</a> Redis will write the shortest sequence of
        commands needed to rebuild the current dataset in memory.  If you're
        using the AOF with Redid 2.2 you'll need to run <a href="/commands/bgrewriteaof">BGREWRITEAOF</a> from time to
        time. Redis 2.4 is able to trigger log rewriting automatically (see the
        2.4 example configuration file for more information).</p>
        
        <h3>How durable is the append only file?</h3>
        
        <p>You can configure how many times Redis will
        <a href="http://linux.die.net/man/2/fsync"><code>fsync</code></a> data on disk. There are
        three options:</p>
        
        <ul>
        <li><p><code>fsync</code> every time a new command is appended to the AOF. Very very
        slow, very safe.</p></li>
        <li><p><code>fsync</code> every second. Fast enough (in 2.4 likely to be as fast as snapshotting), and you can lose 1 second of data if there is a disaster.</p></li>
        <li><p>Never <code>fsync</code>, just put your data in the hands of the Operating
        System. The faster and less safe method.</p></li>
        </ul>
        
        
        <p>The suggested (and default) policy is to <code>fsync</code> every second. It is
        both very fast and pretty safe. The <code>always</code> policy is very slow in
        practice (although it was improved in Redis 2.0) – there is no way to
        make <code>fsync</code> faster than it is.</p>
        
        <h3>What should I do if my AOF gets corrupted?</h3>
        
        <p>It is possible that the server crashes while writing the AOF file (this
        still should never lead to inconsistencies), corrupting the file in a
        way that is no longer loadable by Redis. When this happens you can fix
        this problem using the following procedure:</p>
        
        <ul>
        <li><p>Make a backup copy of your AOF file.</p></li>
        <li><p>Fix the original file using the <code>redis-check-aof</code> tool that ships with
        Redis:</p>
        
        <pre><code>$ redis-check-aof --fix &lt;filename&gt;&#x000A;</code></pre></li>
        <li><p>Optionally use <code>diff -u</code> to check what is the difference between two
        files.</p></li>
        <li><p>Restart the server with the fixed file.</p></li>
        </ul>
        
        
        <h3>How it works</h3>
        
        <p>Log rewriting uses the same copy-on-write trick already in use for
        snapshotting.  This is how it works:</p>
        
        <ul>
        <li><p>Redis <a href="http://linux.die.net/man/2/fork">forks</a>, so now we have a child
        and a parent process.</p></li>
        <li><p>The child starts writing the new AOF in a temporary file.</p></li>
        <li><p>The parent accumulates all the new changes in an in-memory buffer (but
        at the same time it writes the new changes in the old append-only file,
        so if the rewriting fails, we are safe).</p></li>
        <li><p>When the child is done rewriting the file, the parent gets a signal,
        and appends the in-memory buffer at the end of the file generated by the
        child.</p></li>
        <li><p>Profit! Now Redis atomically renames the old file into the new one,
        and starts appending new data into the new file.</p></li>
        </ul>
        
        
        <h3>How I can switch to AOF, if I'm currently using dump.rdb snapshots?</h3>
        
        <p>There is a different procedure to do this in Redis 2.0 and Redis 2.2, as you
        can guess it's simpler in Redis 2.2 and does not require a restart at all.</p>
        
        <p><strong>Redis >= 2.2</strong></p>
        
        <ul>
        <li>Make a backup of your latest dump.rdb file.</li>
        <li>Transfer this backup into a safe place.</li>
        <li>Issue the following two commands:</li>
        <li>redis-cli config set appendonly yes</li>
        <li>redis-cli config set save ""</li>
        <li>Make sure that your database contains the same number of keys it contained.</li>
        <li>Make sure that writes are appended to the append only file correctly.</li>
        </ul>
        
        
        <p>The first CONFIG command enables the Append Only File. In order to do so <strong>Redis will block</strong> to generate the initial dump, then will open the file for writing, and will start appending all the next write queries.</p>
        
        <p>The second CONFIG command is used to turn off snapshotting persistence. This is optional, if you wish you can take both the persistence methods enabled.</p>
        
        <p><strong>IMPORTANT:</strong> remember to edit your redis.conf to turn on the AOF, otherwise
        when you restart the server the configuration changes will be lost and the
        server will start again with the old configuration.</p>
        
        <p><strong>Redis 2.0</strong></p>
        
        <ul>
        <li>Make a backup of your latest dump.rdb file.</li>
        <li>Transfer this backup into a safe place.</li>
        <li>Stop all the writes against the database!</li>
        <li>Issue a redis-cli bgrewriteaof. This will create the append only file.</li>
        <li>Stop the server when Redis finished generating the AOF dump.</li>
        <li>Edit redis.conf end enable append only file persistence.</li>
        <li>Restart the server.</li>
        <li>Make sure that your database contains the same number of keys it contained.</li>
        <li>Make sure that writes are appended to the append only file correctly.</li>
        </ul>
        
        
        <h2>Interactions between AOF and RDB persistence</h2>
        
        <p>Redis >= 2.4 makes sure to avoid triggering an AOF rewrite when an RDB
        snapshotting operation is already in progress, or allowing a BGSAVE while the
        the AOF rewrite is in progress. This prevents two Redis background processes
        from doing heavy disk I/O at the same time.</p>
        
        <p>When snapshotting is in progress and the user explicitly requests a log
        rewrite operation using BGREWRITEAOF the server will reply with an OK
        status code telling the user the operation is scheduled, and the rewirte
        will start once the snapshotting is completed.</p>
        
        <p>In the case both AOF and RDB persistence are enabled and Redis restarts the
        AOF file will be used to reconstruct the original dataset since it is
        guaranteed to be the most complete.</p>
        
        <h2>Backing up Redis data</h2>
        
        <p>Before starting this section, make sure to read the following sentence: <strong>Make Sure to Backup Your Database</strong>. Disks break, instances in the cloud disappear, and so forth: no backups means huge risk of data disappearing into /dev/null.</p>
        
        <p>Redis is very data backup friendly since you can copy RDB files while the
        database is running: the RDB is never modified once produced, and while it
        gets produced it uses a temporary name and is renamed into its final destination
        atomically using rename(2) only when the new snapshot is complete.</p>
        
        <p>This means that copying the RDB file is completely safe while the server is
        running. This is what we suggest:</p>
        
        <ul>
        <li>Create a cron job in your server creating hourly snapshots of the RDB file in one directory, and daily snapshots in a different directory.</li>
        <li>Every time the cron script runs, make sure to call the <code>find</code> command to make sure too old snapshots are deleted: for instance you can take hourly snapshots for the latest 48 hours, and daily snapshots for one or two months. Make sure to name the snapshots with data and time information.</li>
        <li>At least one time every day make sure to transfer an RDB snapshot <em>outside your data center</em> or at least <em>outside the physical machine</em> running your Redis instance.</li>
        </ul>
        
        
        <h2>Disaster recovery</h2>
        
        <p>Disaster recovery in the context of Redis is basically the same story as
        backups, plus the ability to transfer those backups in many different external
        data centers. This way data is secured even in the case of some catastrophic
        event affecting the main data center where Redis is running and producing its
        snapshots.</p>
        
        <p>Since many Redis users are in the startup scene and thus don't have plenty
        of money to spend we'll review the most interesting disaster recovery techniques
        that don't have too high costs.</p>
        
        <ul>
        <li>Amazon S3 and other similar services are a good way for mounting your disaster recovery system. Simply transfer your daily or hourly RDB snapshot to S3 in an encrypted form. You can encrypt your data using gpg -c (in symmetric encryption mode). Make sure to store your password in many differnet safe places (for instance give a copy to the most important guys of your organization). It is recommanded to use multiple storage services for improved data safety.</li>
        <li>Transfer your snapshots using scp (part of ssh) to far servers. This is a fairly simple and safe route: get a small VPS in a place that is very far from you, install ssh there, and greate an ssh client key without passphrase, then make
        add it in the authorized_keys file of your small VPS. You are ready to transfer
        backups in an automated fashion. Get at least two VPS in two different providers
        for best results.</li>
        </ul>
        
        
        <p>It is important to understand that this systems can easily fail if not coded
        in the right way. At least make absolutely sure that after the transfer is
        completed you are able to verify the file size (that should match the one of
        the file you copied) and possibly the SHA1 sum if you are using a VPS.</p>
        
        <p>You also need some kind of independent alert system if the transfer of fresh
        backups is not working for some reason.</p>
      </article>
    </div>
    <div class='text' id='comments'>
      <div id='disqus_thread'></div>
      <script type='text/javascript'>
        //<![CDATA[
          var disqus_shortname = 'rediscn';
          
          /* * * DON'T EDIT BELOW THIS LINE * * */
          (function() {
            var dsq = document.createElement('script'); dsq.type = 'text/javascript'; dsq.async = true;
              dsq.src = 'http://' + disqus_shortname + '.disqus.com/embed.js';
              (document.getElementsByTagName('head')[0] || document.getElementsByTagName('body')[0]).appendChild(dsq);
          })();
        //]]>
      </script>
      <a class='dsq-brlink' href='http://disqus.com'>
        Comments powered by
        <span class='logo-disqus'>
          Disqus
        </span>
      </a>
    </div>
    <script src='/js/foot.js'></script>
  </body>
</html>
